Adaptive rate control for digital video compression

ABSTRACT

A system and method for adaptively controlling the encoded data rate in a data compression system. The system and method sets up alternative encoded bit streams for each segment of data and selects the alternative that would produce the bit rate closest to a predetermined target bit rate for transmission. Each segment of video input is quantized based on a set quantization settings to produce a plurality of quantized segments. Each quantized segment is then variable rate encoded to produce an alternative encoded bit stream. The data rate that would be required to transmit each alternative encoded bit stream is determined and compared with a predetermined target bit rate, which is set according to the transmission rate buffer status. The selected encoded bit stream is provided to the transmission rate buffer in preparation for transmission. Having processed one segment of data, the system and method then updates its parameters for processing the next segment of data. An updated target bit rate is determined based on the rate buffer status at this point. A rate versus quantization index model is derived according to the data rates of the encoded streams and the corresponding quantization indices. A new set of weighting mask functions is then produced in accordance with the model and the quantization indices. The new set of quantization indices is to be used for processing the next segment of data.

This application is a continuation of U.S. patent application Ser. No.08/731,229, filed Oct. 11, 1996, now U.S. Pat. No. 6,366,614 which isincorporated herein by reference in its entirety.

BACKGROUND OF THE INVENTION

I. Field of the Invention

The present invention relates to image processing. More particularly,the present invention relates to a novel and improved system and methodfor adaptively controlling the digital bit rate of compression in avideo encoder.

II. Description of the Related Art

In the field of transmission and reception of television signals,various improvements are being made to the NTSC (National TelevisionSystems Committee) System. Developments in the field of television arecommonly directed towards Standard Definition Television (SDTV) and HighDefinition Television (HDTV) Systems.

Many of the proposed SDTV and HDTV systems make use of digital encodingtechniques. Digitally encoded video offers many advantages over analogmodulation. Digital encoding provides a robustness of the communicationslink to impairments such as multipath and jamming. Furthermore, digitaltechniques facilitate ease in signal encryption, necessary for militaryand many broadcast applications.

When first proposed, HDTV seemed impractical due to excessive bandwidthrequirements. However, it has been realized that compression of digitalHDTV signals may be achieved to a level that enables transmission atbandwidths comparable to that required by analog NTSC formats. Suchlevels of signal compression coupled with digital transmission of thesignal will enable a HDTV system to transmit with less power withgreater immunity to channel impairments.

One compression technique capable of offering significant compressionwhile preserving the quality of SDTV and HDTV signals utilizesadaptively sized blocks and sub-blocks of encoded discrete cosinetransform (DCT) coefficient data. The technique is disclosed in U.S.Pat. No. 5,021,891, entitled “ADAPTIVE BLOCK SIZE IMAGE COMPRESSIONMETHOD AND SYSTEM”, assigned to the assignee of the present inventionand incorporated by reference. DCT techniques are also disclosed in U.S.Pat. No. 5,107,345, entitled “ADAPTIVE BLOCK SIZE IMAGE COMPRESSIONMETHOD AND SYSTEM”, assigned to the assignee of the present inventionand incorporated by reference. Further, U.S. Pat No. 5,452,104, entitled“ADAPTIVE BLOCK SIZE IMAGE COMPRESSION METHOD AND SYSTEM”, is alsoassigned to the assignee of the present invention and incorporated byreference.

Techniques that offer substantial levels of compression often make useof variable-length encoding schemes. In variable-length encoding,different samples of a signal are quantized using different lengths ofcodewords. The coder is generally designed based on the theoretical ormeasured statistics of an image to minimize the overall reconstructionerror. By exploiting the probability distribution of the characteristicsin an image, high compression ratios are achievable.

Although variable-length encoding may provide for high compressionratios, it also causes the complication of a non-constant encoded datarate. Variable-length encoding generally produces long codewords forimage areas with high details, and short codewords for image areas withlow details. When variable-length encoding is used to encode video,different frames of the video may be encoded with different lengths ofcodewords. These codewords need to be transmitted through acommunications channel at a predetermined bit rate. Further, inapplications such as SDTV and HDTV systems, the codewords must betransmitted to the decoder at a rate which will permit forreconstruction of the frames of the video without fluctuations in theframe rate.

A rate buffer has been used to maintain the rate of transmission of theencoded data bits. However, the use of a buffer does not by itself solvethe problem of fluctuations in the decoded frame rate. Further, bufferoverflow may result when one frame of video has been encoded with longcodewords which exceed the capacity of the buffer, resulting in loss ofinformation. Consequently, rate control for video compression isnecessary. These problems and deficiencies are clearly felt in the artand are solved by the present invention in the manner described below.

SUMMARY OF THE INVENTION

The present invention is a novel and improved system and method forcontrolling the encoded data rate in a video compression procedure. Whenvideo is compressed, different segments of the video may be encoded withdifferent lengths of codewords. In order to transmit the codewordsthrough a communications channel at a constant rate while maintainingthe reliability of the encoder, control of the encoded bit rate isnecessary. The present system and method accomplishes rate control bysetting up alternative encoded bit streams for each segment of the videoand selecting the alternative that would produce a bit rate closest to apredetermined target bit rate. The target bit rate is selected based onthe rate buffer status.

In accordance with the present invention, an adaptive data ratecontroller which comprises a plurality of quantizers is disclosed. Therate controller receives as input a block of a video data, and the sameblock of video data is presented to each of the quantizers. Eachquantizer quantizes the samples of the input according to a differentweighting mask function to produce a block of quantized coefficients.Each weighting mask function is identified by a quantization index. Aweighting mask function is designed to emphasize certain samples of theinput and de-emphasize other samples by weighting the samplesdifferently. Thus, the corresponding quantized samples of the differentblocks of quantized coefficients may have different values as a resultof having been weighted differently.

The adaptive rate controller also comprises a plurality of encoders.Each encoder receives one of the blocks of quantized coefficients, andproduces a stream of variable-length encoded coefficients. Because eachblock of quantized coefficients has been processed by a differentweighting function, the samples of each block may be encoded withdifferent lengths of codewords. As a result, each stream ofvariable-length encoded coefficients may have a code length distinctfrom the others.

The variable-length encoded streams are presented to a selector, whilethe total bit rates required for transmitting each of thevariable-length encoded streams are determined and presented to acomparator. The total bit rates are proportional to the sum of the codelengths of the encoded streams. The comparator compares each of thetotal bit rates with a predetermined target bit rate in order todetermine the rate closest to the target. The selector then selects thevariable-length encoded stream which would yield a bit rate closest tothe predetermined target, and presents this stream to a rate buffer inpreparation for transmission.

Now that the current block of video signal has been processed, the ratecontroller prepares to process the next block of video signal byupdating the weighting mask functions. A quantization index updateelement selects a new set of quantization indices from which theweighting mask functions are derived. The new quantization indices areselected based on a model of rate versus quantization index and anupdated target bit rate.

A model element derives the model of rate versus quantization index. Themodel is derived from the rate and quantization index data from thecurrent block of video signal. Thus, the quantization indices used forthe current block of video and the corresponding bit rates are used toderive the model. The updated target bit rate is derived by a target bitrate update element based on the rate buffer fullness level afterprocessing the current block of video input. The updated target bit rateis selected so as to maintain a constant flow of data through the ratebuffer as well as to prevent rate buffer overflow. Based on the modeland the updated target bit rate, one new quantization index is the indexwhich would yield the updated target bit rate as indicated by the model.Other new quantization indices will generally be functions of thealready designated new quantization index.

After the weighting mask functions have been updated, the adaptive ratecontroller of the present invention begins processing the next block ofvideo input.

BRIEF DESCRIPTION OF THE DRAWINGS

The features, objects, and advantages of the present invention willbecome more apparent from the detailed description set forth below whentaken in conjunction with the drawings in which like referencecharacters identify correspondingly throughout and wherein:

FIG. 1 is a block diagram illustrating an exemplary encoding system inwhich a rate controller is utilized;

FIG. 2 is a block diagram illustrating an exemplary preprocessor whichgenerates coefficients for the encoder;

FIG. 3 is a block diagram illustrating the processing elements of theencoder and the rate controller;

FIG. 4 is a graph illustrating an exponential model of quantizationindex versus bit rate;

FIGS. 5 a-5 c are block diagrams illustrating the processing elementswhich select the rate controlled data components of a color signal fortransmission;

FIG. 6 is a block diagram illustrating the processing elements whichderive the quantization indices to be used for quantizing a receivedcolor video signal; and

FIGS. 7 a-7 c are a series of graphs illustrating models of quantizationindex versus bit rate for each of the color components of a colorsignal, and

FIG. 7 d is a graph illustrating a composite model of quantization indexversus bit rate for all components of a color signal.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

An exemplary data compression system which incorporates the ratecontroller of the present invention is illustrated in FIG. 1. The systemshown in FIG. 1 may be used to compress a video signal for transmission.For example, the system of FIG. 1 may be used to compress a HDTV or SDTVsignal, although it should be understood that any other type of video,or even audio, signal may benefit from this compression system.

As shown in FIG. 1, a video signal is first presented to preprocessor 10in preparation for compression. Preprocessor 10 may serve a variety ofpurposes, or may be excluded from the system altogether. Preprocessor 10may, for example, format the video signal into components that are moreeasily processed by the compression system. The output of thepreprocessor 10 is presented to encoder 12. Encoder 12 quantizes thedata that it has received then compresses the quantized coefficients.The quantization scheme performed is dependent on feedback quantizationparameters from rate controller 14. Rate controller 14 utilizesstatistics characterizing the current encoded segment of video toadaptively set the quantization parameters for encoding the next segmentof video. Rate controller 14 also presents the rate controlled encodeddata to formatter 16. Formatter 16 takes the rate controlled data andassembles the data into a formatted bit stream for transmission througha communications channel.

One possible implementation of the preprocessor 10 is illustrated inFIG. 2. As shown in FIG. 2, preprocessor 10 comprises a Two DimensionalDiscrete Cosine Transform (DCT) operator 18. One segment of a videosignal, generally a N×N block of time-sampled pixels, is presented toDCT operator 18 as input. From the block of time-sampled pixels, DCToperator 18 generates a block of DCT coefficients.

DCT operator 18 is one method of converting a time-sampled signal to afrequency representation of the same signal. By converting to afrequency representation, the DCT techniques have been shown to allowfor very high levels of compression, as quantizers can be designed totake advantage of the frequency distribution characteristics of animage. One compression system that utilizes DCT transforms is describedin U.S. Pat. Nos. 5,021,891, 5,107,345, and 5,452,104 mentioned above.

The block of DCT coefficients is presented to encoder 12, with theencoded bit rate controlled by rate controller 14. In an exemplaryembodiment, encoder 12 and rate controller 14 are implemented in amicroprocessor or digital signal processor programmed to provide thefunctions as described.

Referring now to FIG. 3, the details of encoder 12 and rate controller14 are shown. For purposes of illustration, FIG. 3 is described in termsof processing a luminance video signal. Processing of a color videosignal will be described later. Encoder 12 comprises a plurality ofquantizers 20 a-20 c and a corresponding plurality of variable lengthencoders 22 a-22 c. Three sets of quantizers 20 a-20 c and variablelength encoders 22 a-22 c are shown, although it should be understoodthat a different number of elements may be used instead.

Each of the three quantizers 20 a-20 c receives the same block of DCTcoefficients, designated F, as input. Each quantizer 20 a-20 c alsoreceives from rate controller 14 a signal of a feedback quantizationindex, designated by q₁-q₃. In FIG. 3, the three quantizers 20 a-20 crepresent three quantization settings, or three ways of quantizing thesame input signal. The outputs of quantizers 20 a-20 c are blocks ofquantized DCT coefficients, designated QC1-QC3 in FIG. 3.

In a preferred embodiment, the quantization setting used by eachquantizer 20 a-20 c to quantize the input signal is a weighting maskfunction, also known in the art as a quantization matrix. Each weightingmask function is derived by multiplying a selected quantization stepsize (qss_(i)) with the coefficients of a table of frequency weights.The qss_(i) is a function of the quantization index q_(i), such thatqss _(i) =f(q _(i)).  (1)In a preferred embodiment,qss_(i)=2^((q) ^(i) ⁾.  (2)

A table of frequency weights, of the same dimensions as the block ofinput DCT coefficients, is used to apply different weights to thedifferent DCT coefficients. The weights are designed to emphasize theinput samples having frequency content which the human visual system ismore sensitive to, and to de-emphasize samples having frequency contentthat the visual system is less sensitive to. The weights are selectedbased on empirical data. A method for designing the weighting masks for8×8 DCT coefficients is disclosed in ISO/IEC JTC1 CD 10918, “Digitalcompression and encoding of continuous-tone still images—part 1:Requirements and guidelines,” International Standards Organization,1994, which is herein incorporated by reference.

Thus, quantization index q₁ is multiplied with the table of frequencyweighting masks to produce a first weighting mask function. The DCTcoefficients are multiplied with corresponding coefficients of the firstweighting mask function to produce a first block of quantizedcoefficients, designated QC1. Likewise, quantization indices q₂ and q₃are each multiplied with the same table of frequency weighting masks toproduce second and third weighting mask functions in quantizers 20 b and20 c, respectively. Then, the DCT coefficients are multiplied with thecorresponding coefficients of the second weighting mask function toproduce a second block of quantized coefficients, designated QC2. TheDCT coefficients are also multiplied with the corresponding coefficientsof the third weighting mask function to produce a third block ofquantized coefficients, designated QC3. Letting (k, l) refer to theposition of a coefficient within a block, and FWM refer to the table offrequency weighting masks, the operations of quantizers 20 a-20 c may bedescribed by the following equations:QC 1(k, l)=F(k, l)×FWM(k, l)×qss ₁;  (3)QC 2(k, l)=F(k, l)×FWM(k, l)×qss ₂;  (4)QC 3(k, l)=F(k, l)×FWM(k, l)×qss ₃.  (5)

The signals QC1-QC3 are input to variable length encoders 22 a-22 crespectively. The quantized DCT coefficient values are each encodedusing variable length encoders in order to minimize the data rate. Thethree variable length encoders 22 a-22 c shown in FIG. 3 may allimplement the same variable length encoding scheme or may implementdifferent variable length encoding algorithms. The outputs of thevariable length encoders 22 a-22 c are signals of serialized streams ofvariable length encoded coefficients and are designated VC1-VC3.

One technique for implementing variable length encoders 22 a-22 c makesuse of run-length encoding of zeros after zigzag scanning followed byHuffman encoding. This technique is discussed in detail inaforementioned U.S. Pat. Nos. 5,021,891, 5,107,345, and 5,452,104, andis summarized herein. A run-length coder would take the quantizedsignals, in this case QC1-QC3, and separate out the zero from thenon-zero coefficients. The zero values are referred to as run-lengthvalues, and are Huffman encoded. The non-zero values are separatelyHuffman encoded.

Huffman codes are designed from either the measured or theoreticalstatistics of an image. It has been observed that most natural imagesare made up of blank or relatively slowly varying areas, and busy areassuch as object boundaries and high-contrast texture. Huffman coders withfrequency-domain transforms such as the DCT exploit these features byassigning more bits to the busy areas and fewer bits to the blank areas.

Referring still to FIG. 3, it can be seen that the signals VC1-VC3 areinput to corresponding rate measurers 24 a-24 c. Each of rate measurer24 a-24 c determines the bit rate required to transmit the respectivevariable length encoded coefficients of signals VC1-VC3. The output fromeach rate measurer 24 a-24 c is a signal of a single value indicative ofthe bit rate of the block of DCT coefficients. The bit rate isproportional to the number of bits required to variable length encodethe block of DCT coefficients. The signals corresponding to outputs fromrate measurers 24 a-24 c are designated r₁-r₃, respectively.

Two sets of signals are output from encoder 12 to rate controller 14.Rate controller 14 receives the signals of the variable length encodedcoefficients, VC1-VC3. One of VC1-VC3 is to be selected by ratecontroller 14 for transmission. Rate controller 14 also receives signalsr₁-r₃ representative of the bit rates of the variable length encodedcoefficients VC1-VC3. The rate information helps in the selection of thevariable length encoded coefficients. Also, utilizing the rateinformation, rate controller 14 generates updated quantization indicesto be used by quantizers 20 a-20 c in quantizing the next segment ofvideo input. The updated indices are established so as to control thebit rate of the next segment of video input.

As shown in FIG. 3, signals indicative of variable length encodedcoefficients VC1-VC3 are input to selector 28 of rate controller 14,while signals indicative of rates r₁-r₃ are input to comparator 30 andmodel element 32 of rate controller 14. Comparator 30 compares the threerates r₁-r₃ with a desired bit rate in order to choose the rate closestto the desired rate. Based on the chosen rate, comparator 30 provides asignal to selector 28 indicating which one of the streams of variablelength encoded coefficients, VC1, VC2, or VC3, has been selected fortransmission. The function of model element 32 will be described later.

Several selection algorithms may be used to select the stream ofvariable length encoded coefficients for transmission. A preferredembodiment selects the stream that minimizes the absolute encoded rateerror. This method compares a predetermined target bit rate with each ofthe rates r₁, r₂, and r₃ according to the equation:min|T−r _(i)|  (6)where T is the target bit rate and r_(i) for i=1, 2, 3 refers to ratesr₁-r₃, respectively. In an alternative embodiment, selector 28 selectsthe variable length encoded stream that minimizes the rate error andthat has a rate less than the target rate. In a second alternativeembodiment, selector 28 selects the stream that produces the minimumrate.

Selector 28 provides the signal of the stream of variable length encodedcoefficients that has been selected for transmission to rate buffer 34to await transmission through the communications channel. The selectedsignal represents a rate controlled video signal. Then, referring backto FIG. 1, the rate controlled data signal is presented to formatter 16,which formats the data signal with control and identification signals inpreparation for transmission. Signals indicative of start of block,start of frame, block number, frame number, and quantization informationare some of the signals that are appended to the data signal byformatter 16.

At this point, the current segment of the video input has been processedfor transmission. It is now necessary to update the system in order torate controllably encode the next segment of video and prepare the nextsegment for transmission. As rate controller 14 adjusts the encoded bitrate by selecting among three streams of encoded coefficients for eachsegment of video, a new set of three quantization indices needs to bederived.

The new quantization indices are derived by quantization index updateelement 36, shown in FIG. 3. Quantization index update element 36derives the indices based on input signals from model element 32 andtarget bit rate update element 38. Model element 32 derives a model ofencoded bit rate versus quantization index. Target bit rate updateelement 38 derives an updated target bit rate for the next segment ofvideo input. Based on the updated target bit rate and the model ofencoded bit rate versus quantization index, three updated quantizationindices will be selected for quantizing the next segment of video.

Model element 32 derives a model of quantization index versus bit ratefor the next segment of video based on the data of quantization indicesand rates from the current segment of video. Referring still to FIG. 3,it can be seen that model element 32 receives as input signalsindicative of the three quantization indices q₁-q₃ that are used toprocess the current segment of video. Model element 32 also receives asinput signals of the three rates r₁-r₃ corresponding to the rates of thecurrent three streams of variable length encoded coefficients VC1-VC3.From three sets of data points (q₁, r₁), (q₂, r₂), and (q₃, r₃), a modelis derived by fitting a curve through the three data points. In apreferred embodiment, an exponential model is used for the curvefitting.

The exponential model is defined according to the equation:rate_(i)=y_(i)=bm^(x) _(i),  (7)where x_(i) denotes the quantization index, set to range from 0 to 31 ina preferred embodiment, although it should be understood that adifferent range of quantization indices may be used instead. Thecorresponding encoded rate is denoted by rate_(i) (y_(i)). Theparameters b, m of the exponential model can be determined byrecognizing that:ln rate_(t) =ln y _(i) =ln b+x _(t) ln m.  (8)Then, letting Λ denote the set of n (n=3) quantization indices utilizedto encode the current frame of data, the least squares solution to themodel can be defined as: $\begin{matrix}{{{\ln\quad m} = \frac{{n{\sum\limits_{i \in \Lambda}{x_{i}\ln\quad y_{i}}}} - {( {\sum\limits_{i \in \Lambda}x_{i}} )( {\sum\limits_{i \in \Lambda}{\ln\quad y_{i}}} )}}{{n{\sum\limits_{i \in \Lambda}x_{i}^{2}}} - ( {\sum\limits_{i \in \Lambda}x_{i}} )^{2}}},{and}} & (9) \\{{\ln\quad b} = {\frac{{( {\sum\limits_{i \in \Lambda}{\ln\quad y_{i}}} )( {\sum\limits_{i \in \Lambda}{x\quad}_{i}^{2}} )} - {( {\sum\limits_{i \in \Lambda}x_{i}} )( {\sum\limits_{i \in \Lambda}{x_{i}\ln\quad y_{i}}} )}}{{n{\sum\limits_{i \in \Lambda}x_{i}^{2}}} - ( {\sum\limits_{i \in \Lambda}x_{i}} )^{2}}.}} & (10)\end{matrix}$An illustration of an exemplary exponential model is shown in FIG. 4.The exponential model shown in FIG. 4 is derived from the three pairs ofdata points (q₁, r₁), (q₂, r₂), and (q₃, r₃). Signals of the parametersb and m are input to quantization index update element 36.

As previously mentioned, quantization index update element 36 alsoreceives as input signals of an updated target bit rate from target bitrate update element 38. Referring back to FIG. 3, the updated target bitrate is determined by target bit rate update element 38 based on therate buffer status, or the rate buffer fullness level, after processingthe current segment of video input. Rate buffer status indicator 40,coupled to rate buffer 34, determines the rate buffer status, orfullness level, and sends a signal indicative of the status to targetbit rate update element 38.

Let BF denote the rate buffer status. Rate buffer status indicator 40determines the rate buffer status after processing the current segmentof video input (BF_(k)) as follows:BF _(k) =BF _(k−1) +R _(k) −M,  (11)where BF_(k−1) is the rate buffer status before processing the currentsegment, R_(k) is the data bit rate for the current segment, and M isthe fixed transmission bit rate.

Target bit rate update element 38 then determines the updated targetrate, NT_(k), according to the following:NT _(k) =M−α(BF _(k) −γBF _(max)).  (12)where M is again the fixed transmission rate, BF_(max) is the size ofthe rate buffer, α is a constant that determines how fast the ratebuffer converges to the desired rate buffer fullness level, and γ(0.0≦γ≦1.0) is the desired rate buffer fullness level.

In a preferred embodiment, to slow the response of the rate controlsystem in order to prevent fluctuations in the bit rate, a smoothedupdated target rate, SNT_(k), may be derived as follows:SNT _(k) =βNT _(k)+(1−β)SNT _(k−1).  (13)SNT_(k) can be used instead of NT_(k) in the selection process. In apreferred embodiment, α is set to 0.2, and β is set to 0.4.

A signal indicative of the updated target bit rate NT_(k) is presentedto comparator 30 to be used for processing the next segment of videoinput. A signal indicative of the updated target rate NT_(k) is alsopresented to quantization index update element 38 to be used forselecting a set of three updated quantization indices (q₁′-q₃′) to beused by quantizers 20 a-20 c for processing the next segment of videoinput.

Once quantization index update element 36 has received signalsindicative of the updated target bit rate NT_(k) and the parameters band m of the rate versus quantization index model, an updated set ofquantization indices (q₁′-q₃′) may be selected for quantizing the nextsegment of video input.

Quantization index update element 36 may use a number of methods toselect the updated set of quantization indices q₁′-q₃′. A preferredembodiment selects the quantization index q₂′ first. It is determinedaccording to the equation: $\begin{matrix}{{q_{2}^{\prime} = {{round}\quad( \frac{{\ln\quad{NT}_{k}} - {\ln\quad b}}{\ln\quad m} )}},} & (14)\end{matrix}$where the value NT_(k) is the updated target bit rate and the values ofb and m are the parameters of the exponential model described above.

The other two quantization indices, q₁′ and q₃′, may be updatedaccording to either the one-anchor or two-anchor update method. Thesemethods define a spread to be the minimum difference in quantizationindices between any of the three quantization indices q₁′, q₂′, and q₃′.The spread is generally set at 5 for a luminance HDTV signal input. Thespread depends on the spacing between the indices.

The one-anchor method defines an anchor index as A1. Also, it definesq_(max) be the maximum quantization index which equals 31. Assuming that0≦spread≦A1 and 0≦spread≦|q_(max)−A1|, the one-anchor method sets q₃′equal to A1 unless A1 is within spread of q₂′. In that case, q₃′ is setto an index spread away from q₂′. The one-anchor update algorithm isdefined as follows:

-   -   If |q₂′−A1|<spread, then q₁′=q₂′−spread and q₃′=q₁′+spread.    -   If q₂′≧A1+spread and q₂′≦q_(max)−spread, then q₁′=q₂′+spread and        q₃′=A1.    -   If q₂′≦A1−spread and q₂′≧spread, then q₁′=q₂′−spread and q₃′=A1.    -   If q₂′≧A1+spread and q₂′>q_(max)−spread, then q₁′=q₂′−spread and        q₃′=A1.    -   If q₂′≦A1−spread and q₂′<spread, then q₁′=q₂′+spread and q₃′=A1.    -   If q₂′=A1and q_(max)−A1<spread, then q₁′=q₂′−2*spread and        q₃′=q₂′−spread.    -   If q₂′≦q_(max)−spread and q_(max)−A1<spread, then q₁′=q₂′−spread        and q₃′=q₂′+spread.    -   If q₂′=A1 and A1<spread, then q₁′=q₃′+spread and q₃′=q₂′+2*        spread.

The two-anchor method defines two fixed anchors A1 and A2 where A1<A2.The two-anchor method ensures that bit rate overshoots and undershootsare reduced to acceptable levels. Recall that the spread is the minimumdifference in quantization indices between any of the three quantizationindices q₁′, q₂′, and q₃′. Assume that 2 * spread≦A2−A1, spread≦A1, andspread≦|q_(max)−A2|. The two-anchor method sets q₁′ to A1, and sets q₃′to A2 unless A1 or A2 are within spread of q₂′. In these cases eitherq₁′ or q₃′ is set to an index spread away from q₂′. The two-anchorupdate algorithm is defined as follows:

-   -   If |q₂′−A1|<spread and q₂′≧spread, then q₁′=q₂′−spread and        q₃′=A2.    -   If |q₂′−A1|<spread and q₂′<spread, then q₁′=q₂′+spread and        q₃′=A2.    -   If |q₂′−A2|<spread and q₂′≦q_(max)−spread, then q₁′=A1 and        q₃′=q₂′+spread.    -   If |q₂′−A2|<spread and q₂′>q_(max)−spread, then q₁′=A1 and        q₃′=q₂′−spread.    -   If |q₂′−A1|≧spread and |q₂′−A2|≧spread, then q₁′=A1 and q₃′=A2.

In a preferred embodiment, if the middle quantization index q₂′ producestoo many bits, then all quantization indices are increased for the nextframe, thereby decreasing the bit rate for the next block of data. Ifthe middle quantization index q₂′ produces too few bits, then allquantization indices are decreased for the next frame, therebyincreasing the bit rate for the next block of data.

It should be understood that instead of selecting three quantizationindices to process each block of data input, a different number ofindices may be used instead. As previously mentioned, the number ofquantizers may be a number other than three. In this case, acorresponding number of variable length encoders will be needed toencode the quantized coefficients to be provided to the selector, whichthen selects the rate controlled signal from among all encodedcoefficients. Also, a corresponding number of rate measurers willdetermine the data bit rates of the encoded coefficients. The rates areprovided to the comparator which compares all rates with thepredetermined target rate to help in the process of selecting the ratecontrolled signal. The rates are also provided to the model elementwhich derives the quantization index versus bit rate model. The requirednumber of quantization indices are selected from the model. Thus, thedesired encoded bit stream may be selected from any of a predeterminedplurality of encoded bit streams.

Although the present invention has thus far been described primarilywith respect to luminance video signals, it can be appreciated that thepresent invention is equally applicable to color signals. One techniquefor processing color signals is to first convert the signal from RGBspace to YC₁C₂ space, with Y being the luminance, or brightness,component, and C₁ and C₂ being the chrominance, or color, components.Because of the low spatial sensitivity of the eye to color, mostresearchers sub-sample the C₁ and C₂ components by a factor of four inthe horizontal and vertical directions. Two possible YC₁C₂representations are the YIQ representation and the YUV representation,both of which are well known in the art. Referring to FIG. 1, both theRGB to YC₁C₂ conversion (not shown) and sub-sampling (not shown) may beperformed by preprocessor 10.

In a preferred embodiment for processing color video, four luminancecomponents (hereafter designated Y1-Y4) and two chrominance components(hereafter designated C₁ and C₂) are used to represent each segment ofvideo input. There are four luminance components for each chrominancecomponent because each chrominance component is sub-sampled by four. Foreach of the six components, three quantization indices are selected, inorder to produce three blocks of quantized coefficients for eachcomponent. Further, three streams of variable length encodedcoefficients are generated from each set of three blocks of quantizedcoefficients. Rate controller 14 must select one of each set of threestreams for transmission.

FIGS. 5 a-5 c illustrate the elements of rate controller 14 that performthe selection of the streams of variable length encoded coefficients forinputs Y1-Y4, C₁, and C₂. As in the above description for processing aluminance only signal, FIGS. 5 a-5 c show that three alternative encodedsignals are used to select each rate controlled signal. However, itshould be understood that the present invention is equally applicable insystems which generate a different number of alternative encoded signalsfrom which the rate controlled signal is selected.

In the illustrated embodiment of FIGS. 5 a-5 c, each of the fourluminance inputs Y1-Y4 has been quantized based on the same quantizationindices (q₁-q₃) to produce q₁(Y1)−q₁(Y4), q₂(Y1)−q₂(Y4), andq₃(Y1)−q₃(Y4). It should be understood, however, that the differentluminance components Y1-Y4 may be quantized based on differentquantization indices. The quantization is performed by quantizers suchas quantizers 20 a-20 c shown in FIG. 3. Further, each quantizedcomponent is encoded using variable-length encoding, producing signalsdesignated in FIG. 5 a as VC[q₁(Y1)]−VC[q₁(Y4)], VC[q₂(Y1)]−VC[q₂(Y4)],and VC[q₃(Y1)]−VC[q₃(Y4)]. The variable-length encoding is performed byvariable length encoders such as variable length encoders 22 a-22 c inFIG. 3.

The chrominance inputs C₁ and C₂ are also quantized and variable-lengthencoded. The quantization indices for the C₁ and C₂ components are notnecessarily the same, but may be the same, as those used for Y1-Y4. InFIGS. 5 b and 5 c, the encoded C₁ and C₂ components are designated asVC[q₁(C₁)], VC[q₂(C₁)], and VC[q₃(C₁)], and VC[q₁(C₂)], VC[q₂(C₂)], andVC[q₃(C₂)]. The variable-length encoding is performed by variable lengthencoders such as variable length encoders 22 a-22 c in FIG. 3. Just asfor the luminance components, the quantization of the chrominancecomponents is performed by quantizers such as quantizers 20 a-20 c ofFIG. 3, and the variable length encoding is performed by elements suchas variable length encoders 22 a-22 c in FIG. 3.

In processing the luminance components, the variable length encodedcomponents that have been quantized based on the same quantization indexare input to the same rate measurer. As shown in FIG. 5 a, signalsVC[q₁(Y1)]-VC[q₁(Y4)] are input to rate measurer 42 a, signalsVC[q₂(Y1)]-VC[q₂(Y4)] are input to rate measurer 42 b, and signalsVC[q₃(Y1)]-VC[q₃(Y4)] are input to rate measurer 42 c. Accordingly, atotal rate is determined for all luminance components corresponding to aparticular quantization index. Signals of rates determined by ratemeasurers 42 a, 42 b, and 42 c are designated r₁(Y), r₂(Y), and r₃(Y),respectively, and r₁(Y)-r₃(Y) are input to comparator 44. Comparator 44also receives as input a signal of a predetermined target bit rate. In apreferred embodiment, comparator 44 then uses the minimum absolute rateerror selection criterion as described in Equation (6) above to comparerates r₁(Y), r₂(Y), and r₃(Y) with the target bit rate in order tochoose the rate closest to the target bit rate, and a signal of thequantization index corresponding to the chosen rate is presented toselector 46. Selector 46 also receives as input the variable lengthencoded coefficients VC[q₁(Y1-Y4)], VC[q₂(Y1-Y4)], and VC[q₃(Y1-Y4)].Selector 46 then selects the four components of the luminance input thathave been quantized by the selected quantization index, and presentsthese four components to rate buffer 48 for transmission.

Referring to FIG. 5 b, it can be seen that the C₁ chrominance componentis processed in a manner similar to the luminance components, excepteach of rate measurers 50 a-50 c need only to determine the data rate ofa single C₁ component, not four luminance components. Rate measurer 50 areceives as input the variable length encoded coefficients VC[q₁(C₁)]corresponding to quantization index q₁, and generates data rate r₁(C₁)as output. Similarly, rate measurers 50 b and 50 c receive as inputsvariable-length coefficients VC[q₂(C₁)] and VC[q₃(C₁)] corresponding toquantization indices q₂ and q₃, respectively, and generate data ratesr₂(C₁) and r₃(C₁) as output. Signals indicative of rates r₁(C₁)-r₃(C₁)are input to comparator 52. Comparator 52 may again use the minimumabsolute rate error selection criterion of Equation (6) above to choosethe rate closest to the target bit rate, and present to selector 54 asignal of the quantization index corresponding to the chosen rate.Selector 54 then selects from the three C₁ components,VC[q₁(C₁)]-VC[q₃(C₁)], the one C₁ component quantized by the selectedquantization index to present to rate buffer 56 for transmission.

Referring now to FIG. 5 c, it can be seen that the C₂ component is alsoprocessed in a manner similar to the C₁ component. Again, each ratemeasurer 58 a-58 c determines the data rate associated with each ofthree variable-length encoded streams VC[q₁(C₂)], VC[q₂(C₂)], andVC[q₃(C₂)]. The rates, designated r₁(C₂)-r₃(C₂), are input to comparator60. In a preferred embodiment, comparator 60 chooses from ratesr₁(C₂)-r₃(C₂) the rate closest to the target bit rate based on theabsolute rate error selection criterion of Equation (6) above, andpresents to selector 62 a signal of the quantization index correspondingto the chosen rate. Selector 62 selects the variable length encodedcoefficients produced according to the selected quantization index, andpresents the signal of the selected coefficients to rate buffer 64 fortransmission.

It should be understood that although FIGS. 5 a-5 c show comparators 44,52, and 60 as three separate blocks, the three blocks may be combined sothat a single comparator receives as input all of the signalsr₁(Y)-r₃(Y), r₁(C₁)-r₃(C₁), and r₁(C₂)-r₃(C₂). The single comparatoralso receives as input a total target bit rate. Likewise, selectors 46,54, and 62 may be combined as a single selector which receives as inputthe selected index or indices from the single comparator, and receivesas input all variable-length coefficients VC[q₁(Y1)]-VC[q₁(Y4)],VC[q₁(C₁)]-VC[q₃(C₁)], and VC[q₁(C₂)]-VC[q₃(C₂)]. In addition, thesingle selector may send all selected components to one combined ratebuffer rather than the three rate buffers 48, 56, and 64.

The single comparator and single selector may use a number of decisionrules to select the streams of variable-length coefficients fortransmission. In a preferred embodiment, the 27 possible combinations ofthe sum r_(i)(Y)+r_(j)(C₁)+r_(k)(C₂), (1≦i≦3, 1≦j≦3, 1≦k≦3) are eachcompared with the total target bit rate. The sum which is closest to thetotal target bit rate, or which minimizes the absolute encoded rateerror, is determined. Then, a signal indicative of the threequantization indices corresponding to the sum is provided to the singleselector, which selects the components that have been quantized by thethree quantization indices, and presents these components to the ratebuffer in preparation for transmission. As an example, rates r₂(Y),r₁(C₁), and r₁(C₂) may sum up to a value which is closest to the totaltarget bit rate. The single comparator thus presents the quantizationindices q₂ for the Y components, q₁ for the C₁ component, and q₁ for theC₂ component to the single selector. The single selector then selectsthe signals VC[q₂(Y1-Y4)], VC[q₁(C₁)], and VC[q₁(C₂)] and presents thesesignals to the rate buffer for transmission.

Just as for luminance video, a rate controller for processing colorvideo needs to update the quantization indices for processing the nextsegment of video. Again, the updated indices are based on a model ofquantization index versus rate derived from the quantization index andrate data of the current segment of video. Referring now to FIG. 6,quantization index update element 66 produces quantization indicesq₁′-q₃′ for the next segment of video based on two inputs. One input isa signal of the updated target bit rate. In a preferred embodiment, thetarget bit rate is a total target bit rate for all luminance andchrominance components, produced by target total bit rate update element68. Another input is a signal of the parameters (b and m) of the modelof rate versus quantization index derived by model element 70. In thepreferred embodiment, model element 70 derives a composite model for allluminance and chrominance components.

The operation of the preferred embodiment of model element 70 may bebetter understood with reference to FIGS. 7 a-7 d in conjunction withFIG. 6. As shown in FIG. 6, model element 70 comprises three componentmodel elements designated Y model element 72, C₁ model element 74, andC₂ model element 76. The three component elements derive componentmodels from which a composite model is derived by composite modelelement 78.

Each component model element derives an exponential model ofquantization index versus rate for the component based on thequantization indices used and their corresponding rates from the currentsegment of input. Y model element 72 receives as input three sets ofdata points (q₁(Y), r₁(Y)), (q₂(Y), r₂(Y)), and (q₃(Y), r₃(Y)),designated by (q_(i)(Y), r_(i)(Y)) in FIG. 6. The data rates r_(i)(Y)refer to the total rates from encoding all four luminance components ofone segment of video input for the three quantization indices. The threesets of data points are used to derive the Y-component exponential modelillustrated in FIG. 7 a, according to Equations (7)-(10) above.

C₁ model element 74 receives as input three sets of data points (q₁(C₁),r₁(C₁)), (q₂(C₁), r₂(C₁)), and (q₃(C₁), r₃(C₁)), designated by(q_(i)(C₁), r_(i)(C₁)) in FIG. 6. The data points are the quantizationindices used for the C₁ component and the data rates resulting from theuse of the quantization indices. The three sets of data points are usedto derive the C₁-component exponential model illustrated in FIG. 7 b,again according to Equations (7)-(10) above. Then, based on the model,rates at the current quantization indices for the Y component areestimated. In FIG. 7 b, q₂(C₁) has been set to equal to q₂(Y), so r₂(C₁)based on q₂(Y) remains r₂(C₁). However, estimates are derived for r₁(C₁)based on q₁(Y) and r₃(C₁) based on q₃(Y).

Similarly, C₂ model element 76 receives as input three sets of datapoints (q₁(C₂), r₁(C₂)), (q₂(C₂), r₂(C₂)), and (q₃(C₂), r₃(C₂)),designated by (q_(i)(C₂), r_(i)(C₂)) in FIG. 6. The data points are thequantization indices used for the C₂ component and the data ratesresulting from the use of the quantization indices. These three sets ofdata points are used to derive the C₂-component exponential modelillustrated in FIG. 7 c, according to Equations (7)-(10) above. Also,rates at the current quantization indices for the Y component areestimated based on the C₂-component model. Thus, r₁(C₂) based on q₁(Y),r₂(C₂) based on q₂(Y), and r₃(C₂) based on q₃(Y) are estimated. In FIG.7 b, q₂(C₂)=q₂(Y), thus r₂(C₂) based on q₂(Y) remains r₂(C₂).

From the three component models, a composite model is derived. Thecomposite model is based on three sets of data points: (q₁(Y),r₁(total)), (q₂(Y), r₂(total)), and (q₃(Y), r₃(total)). The total ratesare determined according to the following equations:r ₁(total)=r ₁(Y)+r ₁(C ₁) based on q ₁(Y)+r ₁(C ₂) based onq₁(Y);  (15)r ₂(total)=r ₂(Y)+r ₂(C ₁) based on q ₂(Y)+r ₂(C ₂) based onq₂(Y);  (16)r ₃(total)=r ₃(Y)+r ₃(C ₁) based on q ₃(Y)+r ₃(C ₂) based onq₃(Y);  (17)The composite model is illustrated in FIG. 7 d. Again, an exponentialmodel according to Equations (7) is used to derive the model. Further,the least squares solution to the model as defined in Equations (9)-(10)is used to derive the model parameters b(total) and m(total). Asmentioned above, signals of b(total) and m(total) are input toquantization index update element 66.

The other input to quantization index update element 66, the updatedtarget total bit rate, is derived by target total bit rate updateelement 68 based on the rate buffer status after processing the currentsegment of input signal. Referring back to FIGS. 5 a-5 c, it can be seenthat the selected Y, C₁, and C₂ components are presented to rate buffers48, 56, and 64 prior to transmission through the communications channel.As described above, although rate buffers 48, 56, and 64 are shown asthree separate rate buffers, it should be understood that there may bejust one rate buffer shared among all components. In the embodimentillustrated in FIG. 6, a single rate buffer 80 receives all selected Y,C₁, and C₂ components. Rate buffer status indicator 82 then determinesthe rate buffer status (BF_(k)) after processing the current segment ofvideo input based on Equation (11) above. The rate buffer status BF_(k)is provided to target total bit rate update element 68, which derivesthe updated target bit rate NT_(k) based on Equation (12) above. Targettotal bit rate update element 68 then provides the updated target bitrate NT_(k) to quantization index update element 66.

Having obtained inputs of b(total), m(total), and updated target totalbit rate, quantization index update element 66 is now ready to selectupdated quantization indices for processing the next segment of videoinput. Three quantization indices are to be selected for each Y, C₁, andC₂ component. For the Y component, the updated quantization indices willbe referred to as q₁′(Y)-q₃′(Y), for the C₁ component, the updatedquantization indices will be referred to as q₁′(C₁)-q₃′(C₁), and for theC₂ component, the updated quantization indices will be referred to asq₁′(C₂)-q₃′(C₂).

In a preferred embodiment, the update algorithm selects the same middlequantization index for all Y, C₁, and C₂ components. The quantizationindex q₂′(Y) is selected according to Equation (14) above. The middlequantization indices for the C₁ and C₂ components, q₂′(C₁) and q₂′(C₂)are set to equal q₂′(Y). Then, the two-anchor method described above isused to set q₁′(Y) and q₃′(Y) for all luminance components. The other C₁and C₂ quantization indices are selected so that they are different andhave an absolute difference of spread from the q₂′(C₁) and q₂′(C₂)quantization indices, respectively. These updated quantization indicesare then presented to encoder 12 (FIG. 1) to be used for quantizing thenext segment of input signal.

The previous description of the preferred embodiments is provided toenable any person skilled in the art to make or use the presentinvention. The various modifications to these embodiments will bereadily apparent to those skilled in the art, and the generic principlesdefined herein may be applied to other embodiments without the use ofthe inventive faculty. Thus, the present invention is not intended to belimited to the embodiments shown herein but is to be accorded the widestscope consistent with the principles and novel features disclosedherein.

1. In an image compression system, a sub-system for adaptivelycontrolling the encoded bit rate of a video signal, comprising: encodermeans for receiving a segment of video signal and generating a pluralityof encoded streams of data bits in accordance with a predetermined setof quantization settings, said encoder means comprising rate measurermeans for receiving said plurality of encoded streams and fordetermining a bit rate corresponding to each of said plurality ofencoded streams, said bit rates being dependent on said quantizationsettings; and rate controller means for receiving said plurality ofencoded streams and said corresponding bit rates, said rate controllermeans comprising a selector means for generating a selected encodedstream in accordance with a predetermined selection algorithm based onsaid bit rates.
 2. The sub-system of claim 1 wherein said encoder meanscomprises: a plurality of quantizers, each of said quantizers forreceiving said segment of data input and for quantizing said segment ofdata input in accordance with one of said quantization settings togenerate a quantized stream of data; a plurality of variable lengthencoders, each for receiving one of said quantized streams and forencoding said received quantized stream in accordance with a variablelength encoding algorithm to generate one of said encoded streams ofdata bits.
 3. The sub-system of claim 1 wherein said rate controllermeans comprises update means for generating an updated set of saidquantization settings in accordance with the status of said ratecontroller means after generating said selected encoded stream.
 4. Thesub-system of claim 1 wherein said rate controller means furthercomprises: a buffer for receiving said selected encoded stream inpreparation for transmission, said buffer having a buffer status as aresult of having received said selected encoded stream; and update meansfor generating an updated set of said quantization settings inaccordance with said buffer status.
 5. The sub-system of claim 2 whereinsaid variable length encoding algorithm comprises Huffman coding.
 6. Thesub-system of claim 2 wherein said variable length encoding algorithmcomprises run-length encoding of zeros followed by Huffman coding. 7.The sub-system of claim 4 wherein each of said quantization settingscomprises a weighting mask function generated in accordance with aquantization index.
 8. The sub-system of claim 1 wherein saidpredetermined selection criterion used by said selector means minimizesan absolute encoded rate error based on a predetermined target bit rate.9. The sub-system of claim 1 further comprising a preprocessor forreceiving a block of pixel data and for performing a discrete cosinetransform (DCT) operation on said block of pixel data to generate ablock of DCT coefficient values which is presented to said encoder assaid segment of video signal.
 10. The sub-system of claim 1 wherein saidvideo signal is a color video signal.
 11. A method for adaptivelycontrolling the encoded bit rate of a video input for image compression,comprising the steps of: encoding a segment of video in accordance witha predetermined set of quantization settings to generate a plurality ofencoded streams of data bits, selecting a selected encoded stream inaccordance with a predetermined selection algorithm based on a bit rate;generating a control signal based on said selected encoded stream; andupdating said quantization settings in accordance with said controlsignal.
 12. The method of claim 11 wherein said step of encodingcomprises the steps of: quantizing said segment of video signal togenerate a plurality of quantized streams, each of said plurality ofquantized streams quantized according to one of said quantizationsettings; and variable-length encoding each of said plurality ofquantized streams using a variable-length encoding algorithm to generatea corresponding plurality of said encoded streams.
 13. The method ofclaim 12 further comprising the steps of: determining an encoded bitrate for each of said plurality of encoded streams; storing saidselected encoded stream in a buffer in preparation for transmission; andwherein said predetermined selection criterion is based on said encodedbit rates; and wherein said control signal is a buffer status signalindicative of the status of said buffer after receiving said selectedencoded stream.
 14. The method of claim 12 wherein said variable lengthencoding algorithm comprises Huffman coding.
 15. The method of claim 12wherein said variable length encoding algorithm comprises run-lengthencoding of zeros followed by Huffman coding.
 16. The method of claim 13wherein each of said quantization settings is a weighting mask functiongenerated in accordance with a quantization index.
 17. The method ofclaim 12 further comprising the step of receiving a block of pixel dataand performing a discrete cosine transform (DCT) operation on saidsegment of pixel data to generate a block of DCT coefficients which issaid segment of video signal.
 18. A method for adaptively controllingthe encoded bit rate of a video input for image compression, comprisingthe steps of: encoding a segment of video in accordance with apredetermined set of quantization settings to generate a plurality ofencoded streams of data bits; determining a bit rate corresponding toeach of said plurality of encoded streams; and selecting a selectedencoded stream in accordance with a predetermined selection algorithmbased on said bit rates corresponding to said plurality of encodedstreams.
 19. In an image compression system, a sub-system forcontrolling the encoded bit rate of a video signal, comprising: meansfor receiving a segment of video signal and generating a plurality ofencoded streams of data bits and corresponding bit rates; and means forreceiving said plurality of encoded streams and corresponding bit rates,comprising: means for receiving the bit rates and comparing the bitrates with a target bit rate; and means for receiving said plurality ofencoded streams and selecting one of said plurality of encoded streamsbased on the results of the comparison.
 20. The sub-system of claim 19wherein said means for receiving the segment of video segment generatessaid plurality of encoded streams of data bits in accordance withpredetermined set of quantization indices and wherein said means forreceiving said plurality of encoded streams further comprises: means forreceiving the bit rates and deriving model of bit rate versusquantization index; means for deriving an updated target bit rate; andmeans for deriving an updated set of quantization indices for a nextsegment of video signal based on the updated target bit rate and modelof bit rate versus quantization index.
 21. In an image compressionsystem, a sub-system for controlling the encoded bit rate of a videosignal, comprising: means for receiving a segment of video signal; andmeans for receiving said plurality of encoded streams; wherein the meansfor receiving the segment of video signal comprises: a plurality ofmeans, each for receiving and quantizing the segment of video signal inaccordance with a predetermined quantization index; a plurality ofmeans, each for receiving and encoding one of quantized segment of videoto generate a plurality of encoded streams of data bits; and wherein themeans for receiving said plurality of encoded streams comprises: meansfor generating a selected encoded stream in accordance withpredetermined selection algorithm; and a buffer for receiving saidselected encoded stream in preparation for transmission.